In conventional storage systems, RAID (redundant array of independent disks) based data protection can be provided by individual RAID groups. However, actual physical disks must be present and selected to act as a RAID group before the corresponding storage space can be made available for use. When a disk fails in a conventional RAID system, the failed disks must be quickly replaced, either by using a hot spare or by manually replacing the failed disk. Once the failed disk is swapped with a replacement disk, a period of high frequency I/O is directed to the replacement disk to reconstruct the data stored thereon in order to provide ongoing data protection. Namely, the RAID group enters a degraded state until the missing data from the failed disk is reconstructed on the replacement disk. During the updating period of the replacement disk, the RAID group is vulnerable to subsequent disk failures.
Another problem also exists in conventional storage systems that rely on RAID based data protection. In the event of power failure, partially written RAID stripes cannot be recovered. In general, to overcome this problem, uninterruptible power supplies have been provided or memory areas having battery backup protection have been provided. In addition, conventional RAID based storage systems are inflexible since all disks in a RAID group are dedicated to a single level of protection regardless of the storage utilization, or lack thereof, in the disks. Formatting a RAID group is a time consuming process that can further contribute to the delay of utilizing the storage space therein. While providing a hot spare can provide a ready replacement disk, such configurations require that one or more disks in the storage system remain idle and unused until a failure occurs. On the other hand, if no hot spare is provided, careful attention must be paid to the RAID group to ensure that when a failure does occur, that prompt replacement of the failed disk occurs.
When in the process of recovering a failed disk by restoring data to the replacement disk, all writes are directed to the replacement disk in order to restore the protection level of the particular RAID group. During this time, the RAID group is susceptible to additional disk failures and the time until the protection level of the RAID group is restored is generally limited by the bandwidth of the head assembly that writes to the replacement disk.
In addition, object storage systems store objects, such as files, data structures, or other types of objects, as well as information about the objects, referred to as metadata. Metadata for files and other objects may include the location of the object data within a corresponding block storage service, as well as information about creation and access times, object creator and owner, access permissions, storage size, file type, and so forth. Additionally, metadata may often be stored in a database that is essentially an object of a fixed size.
As one example, in traditional computer file systems, a user setting up the object storage system may typically reserve a fixed number of regions of a disk for storing object metadata (e.g., UNIX inodes) such as at the time of file system initialization. Accordingly, the user setting up a file system must anticipate the number of and size of the files that the file system will hold before placing the file system into service. For example, the user may err by anticipating a smaller number of files than are actually created subsequently, and may under-provision the amount of metadata space. For instance, after the metadata space has been consumed, no more data can be stored in the file system. Alternatively, the user may err by anticipating a larger number of files than are actually created subsequently, and may overprovision the metadata space. The unused metadata space in this case is unavailable for storing data and therefore resources are wasted.
Similarly, other types of computer object storage systems may customarily allocate a region of fixed storage for a database of metadata entries for the objects being stored. Accordingly, if the storage is used for a larger number of objects than anticipated, there exists the risk of underprovisioning and being unable to store additional objects. Alternatively, the user may err by anticipating a larger number of objects, and this can result in reserved metadata space being unused when a smaller number of objects than expected is placed in the storage.
Furthermore, a plurality of disks, such as hard disk drives (HDDs) and/or solid state drives (SSDs) may be stored in a tray. For example, data may be stored across a plurality of disks located in a single tray. In addition, system software that provides interface and computing capabilities to a computing device, such as an array controller or storage node, may also be stored in the disks located in a tray. Failure of the storage infrastructure in a particular tray may make the data stored in the system unavailable. As one example, failure of the power modules in a tray may make the entire system unavailable, e.g., if OS or other system modules are stored on the tray. Furthermore, the use of single trays for allocating storage for volumes, or the like, may not provide sufficient protection to assure data availability for a system.